Linux System Administrator's Survival Guide lsg26.htm

• Creating and Running Shell Programs
• Using Variables
  • Assigning a Value to a Variable
  • Understanding Positional Parameters and Other Built-In Shell Variables
• Using Quotation Marks
• Using the test Command
• Using Conditional Statements
  • The if Statement
  • The case Statement
• Using Iteration Statements
  • The for Statement
  • The while Statement
  • The until Statement
  • The shift Command
  • The select Statement
  • The repeat Statement
• Using Functions
• Summary

Shell programming is one of the most useful tools a system administrator has. The ability to write a short program to complete an otherwise time-consuming task is much more powerful than knowing every Linux administration tool in detail. Shell programming can make a system administrator's life so much easier that it should be a mandatory skill.

If these statements sound a little too good to be true, consider the many tasks system administrators face every day involving multiple files or directories. Whenever you deal with a number of files, shell programming can make your job easier. This chapter can't show you much more than the basics, but they should help you in your daily work. Lots of excellent books on shell programming are available, so be sure to keep one handy.

Creating and Running Shell Programs

At the simplest level, shell programs are just files that contain one or more shell or Linux commands. You can use these programs to simplify repetitive tasks, to replace two or more commands that are always executed together with a single command, to automate the installation of other programs, and to write simple interactive applications.

To create a shell program, you must create a file using a text editor and put the shell or Linux commands that you want to be executed into that file. Suppose that you have a CD-ROM drive mounted on your Linux system. This CD-ROM device is mounted when the system is first started. If you change the CD that is in the drive at a later time, you must force Linux to read the new directory contents. One way of achieving this task is to put the new CD into the drive, unmount the CD-ROM drive using the umount command, and then remount the drive using the mount command. The following commands show this sequence of steps:

umount /dev/cdrom

mount /dev/cdrom /cdrom

Instead of typing both of these commands each time you change the CD, you can create a shell program that executes both of these commands for you. To create this shell program, put the two commands into a file and call the file remount (or any other name you want).

There are several ways of executing the commands in the remount file. One way is to make the file executable by entering the following command:


chmod +x remount

This command changes the permissions of the file so that it is executable. To run your new shell program, type remount on the command line.

---

The remount shell program must be in a directory that is in your search path or the shell will not be able to find the program to execute. If you can't run the command because it isn't found, specify the path.
Also, if you are using tcsh to write programs, the first line of the shell program must start with a # in order for tcsh to recognize it as a tcsh program file. In fact, it is safest to make sure the first line of every shell program is #!/bin/sh to make sure the shell program is executed as a Bourne shell process. This prevents many problems with the C shell trying to interpret Bourne shell syntax.

---

Another way that you can execute the shell program is to run the shell that the program was written for and pass the program as a parameter to the shell. In the case of a tcsh program, you enter the following command:

tcsh remount

This command starts up a new shell and tells it to execute the commands in the remount file.

A third way of executing the commands that are in a shell program file is to use the . (dot) command (with both the pdksh and bash shells) or the source command in the tcsh shell. These commands tell the shell to execute the file that is passed as an argument. For example, you can use the following command to tell bash or pdksh to execute the commands in the remount file:

. remount

To do the same thing in tcsh, use the following command:

source remount

The following example shows another situation in which a simple shell program can save a lot of time. Suppose that you were working on three different files in a directory every day, and you wanted to back up those three files onto a floppy disk at the end of each day. To do this task, you type in a series of commands:
mount -t msdos /dev/fd0 /a

cp file1 /dev/fd0

cp file2 /dev/fd0

cp file3 /dev/fd0

One way of backing up the files is to mount the floppy disk and then type three copy commands, one for each file that you want to copy. A simpler way is to put the four commands into a file called backup and then execute the backup command when you want to copy the three files onto the floppy disk drive.

---

You still have to ensure the backup shell program is executable and is in a directory that is in your path before you run the command. Also, be careful about using a filename that corresponds to a command name. If there is a program called backup, for example, in the shell's search path before it reads the current directory, that command would execute instead of the shell command file. For this reason, try to use filenames for your shell scripts that are not close to Linux commands.

---

Using Variables

As is the case with almost any language, the use of variables is very important in shell programs. You have seen several types of variables before, of course. Some examples of commonly used variables are the PATH variable and the TERM variable. These variables are examples of built-in shell variables, which are variables that are defined by the shell program that you are using. This section describes how you can create your own variables and use them in simple shell programs.

Assigning a Value to a Variable

In all three of the shells supplied with Linux (Bourne, Korn, and C shell variants), you can assign a value to a variable by typing the variable name followed by an equal sign and then typing the value that you want to assign to the variable. For example, to assign a value of five to the variable named count, enter the following command in bash or pdksh:


count=5

With tcsh, enter the following command to achieve the same results:

set count = 5

---

When setting a variable for the bash and pdksh shells, make sure that there are no spaces on either side of the equal sign. With tcsh, spaces do not matter.

---

Because the shell language is a non-typed interpretive language, you do not have to declare the variable as you would if you were programming in C or Pascal. You can use the same variable to store character strings or integers. You store a character string into a variable in the same way you store an integer into a variable, as shown in the following example:

name=Garry (for pdksh and bash)

set name = Garry (for tcsh)

After you store a value into a variable, how do you get the value back out? You precede the variable name with a dollar sign ($).To print the value that stored in the count variable to the screen, enter the following command:

echo $count

If you omit the $ from the preceding command, the echo command displays the word count on-screen.

Understanding Positional Parameters and Other Built-In Shell Variables

When you run a shell program that requires or supports a number of command line options, each of these options is stored into a positional parameter. The first parameter is stored into a variable named 1, the second parameter is stored into a variable named 2, and so on. The shell reserves these variable names so you cannot use them as variables that you define. To access the values that are stored in these variables, you must precede the variable name with a dollar sign ($) just as you do with variables that you define.

The following shell program expects to be invoked with two parameters. The program takes the two parameters and prints the second parameter that was typed on the command line first and the first parameter that was typed on the command line second:

#program reverse, prints the command line parameters out in reverse #order

echo "$2"

echo "$1"

If you invoked this program by entering the command

reverse hello there

the program would return the output

there hello

A number of other built-in shell variables are important to know about when you are doing a lot of shell programming. Table 26.1 lists these variables and gives a brief description of what each is used for.

Table 26.1. Built-in shell variables.

Using Quotation Marks

The use of the different types of quotation marks is very important in shell programming. The shell uses both kinds of quotation marks and the backslash character to perform different functions. The double quotation marks (""), the single quotation marks (''), and the backslash (\) are all used to hide special characters from the shell. The back quotes have a special meaning for the shell and should not be used to enclose strings. Each of these methods hide varying degrees of special characters from the shell.

The double quotation marks are the least powerful of the three methods. When you surround characters with double quotes, all the whitespace characters are hidden from the shell, but all other special characters are still interpreted. This type of quoting is most useful when you are assigning strings that contain more than one word to a variable. For example, to assign the string hello there to the variable called greeting, enter the following commands:

greeting="hello there" (in bash and pdksh)

set greeting = "hello there" (in tcsh)

This command stores the whole hello there string into the greeting variable as one word. If you typed in this command without using the quotes, bash and pdksh wouldn't understand the command and would return an error message, and tcsh would assign the value hello to the greeting variable and ignore the rest of the command line.

Single quotes are the most powerful form of quoting. They hide all special characters from the shell. This type of quoting is useful if the command you enter is intended for a program other than the shell. You can, for example, use single quotes to write the hello there variable assignment, but you can't use this method in some instances. If the string being assigned to the greeting variable contains another variable, for example, you have to use double quotes. Suppose you wanted to include the name of the user in your greeting. You would type the following commands:

greeting="hello there $LOGNAME" (for bash and pdksh)

set greeting="hello there $LOGNAME" (for tcsh)

---

The LOGNAME variable is a shell variable that contains the Linux username of the person that is logged on to the system.

---

These commands stores the value hello there root into the greeting variable if you are logged into Linux as root. If you try to write this command using single quotes, the single quotes hide the dollar sign from the shell, and the shell doesn't know that it is supposed to perform a variable substitution. As a result, the greeting variable is assigned the value of hello there $LOGNAME.

Using the backslash is the third way of hiding special characters from the shell. Like the single quotation mark method, the backslash hides all special characters from the shell, but it can hide only one character at a time, as opposed to groups of characters. You can rewrite the greeting example using the backslash instead of double quotation marks by using the following commands:


greeting=hello\ there (for bash and pdksh)

set greeting=hello\ there (for tcsh)

In this command, the backslash hides the space character from the shell and the string hello there is assigned to the greeting variable.

Backslash quoting is used most often when you want to hide only a single character from the shell. This situation occurs when you want to include a special character in a string. For example, to store the price of a box of computer disks into a variable named disk_price, use the following command.

disk_price=\$5.00 (for bash and pdksh)

set disk_price = \$5.00 (tcsh)

The backslash in this example hides the dollar sign from the shell. If the backslash were not there, the shell would try to find a variable named 5 and perform a variable substitution on that variable. If there were no variables named 5 defined, the shell would assign a value of .00 to the disk_price variable. (This shell would substitute a value of null for the $5 variable.) You could also use single quotes in the disk_price example to hide the dollar sign from the shell.

The back quote marks (``) perform a different function. You use them when you want to use the results of a command in another command. For example, to set the value of the contents variable to be equal to the list of files that are in the current directory, type the following command:

contents=`ls` (for bash and pdksh)

set contents = `ls` (for tcsh)

This command executes the ls command and stores the results of the command into the contents variable. As shown later in the iteration statements section, this feature can be very useful when you want to write a shell program that performs some action on the results of a another command.

Using the test Command

In bash and pdksh, the test command is used to evaluate conditional expressions. You typically use the test command to evaluate a condition in a conditional statement or to evaluate the entrance or exit criteria for an iteration statement. The test command has the following syntax:


test expression

or


[ expression ]

You can use several built-in operators with the test command. These operators are classified into four different groups: string operators, integer operators, file operators, and logical operators.

You use the string operators to evaluate string expressions. Table 26.2 lists the string operators that the three shell programming languages support.

Table 26.2. String operators for the test command.

The shell integer operators perform similar functions to the string operators except that they act on integer arguments. Table 26.3 lists the test command's integer operators.

Table 26.3. Integer operators for the test command.

You use the test command's file operators to perform functions such as checking to see whether a file exists and checking to see what kind of file the file passed as an argument to the test command is. Table 26.4 lists the test command's file operators.

Table 26.4. File operators for the test command.

You use the test command's logical operators to combine integer, string, or file operators or to negate a single integer, string, or file operator. Table 26.5 lists the test command's logical operators.

Table 26.5. Logical operators for the test command.

The tcsh shell does not have a test command, but tsch expressions perform the same function. The expression operators that tcsh supports are almost identical to those supported by the C language. You use these expressions mostly in if and while commands. Later in this chapter, the sections "Using Conditional Statements" and "Using Iteration Statements" cover these commands. Like the bash and pdksh test command, tcsh expressions support integer, string, file, and logical operators. Table 26.6 lists the integer operators supported by tcsh expressions.

Table 26.6. Integer operators for tcsh expressions.

Table 26.7 lists the string operators that tcsh expressions support.

Table 26.7. String operators for tcsh expressions.

Table 26.8 lists the file operators that tcsh expressions support.

Table 26.8. File operators for tcsh expressions.

Table 26.9 lists the logical operators that tcsh expressions support are listed.

Table 26.9. Logical operators for tcsh expressions.

Using Conditional Statements

The bash, pdksh, and tcsh shells each have two different forms of conditional statements, the if statement and the case statement. You use these statements to execute different parts of your shell program depending on whether certain conditions are true. As with most statements, the syntax for these statements is slightly different between the different shells.

The if Statement

All three shells support nested if-then-else statements. These statements provide you with a way of performing complicated conditional tests in your shell programs. The syntax of the if statement in bash and pdksh is the same:

if [ expression ]

then

 commands

elif [ expression2 ]

 commands

else

 commands

fi

Note that bash and pdksh use the reverse of the statement name in most of their complex statements to signal the end of the statement. In the preceding statement, the fi key word is used to signal the end of the if statement.

The elif and else clauses are both optional parts of the if statement. The elif statement is an abbreviation for else if. This statement is executed only if none of the expressions associated with the if statement or any elif statements before it were true. The commands associated with the else statement are executed only if none of the expressions associated with the if statement or any of the elif statements were true.

In tcsh, the if statement has two different forms. The first form provides the same function as the bash and pdksh if statement. This form of if statement has the following syntax:

if (expression1) then

 commands

else if (expression2) then

 commands

else

 commands

endif

Once again the else if and else parts of the if statement are optional. This statement could have been written with an elif, as well. If the preceding code shown is the entire tcsh program, it should begin with the following line to make sure it runs properly:

#!/bin/sh

The second form of if statement that tcsh provides is a simple version of the first if statement. This form of if statement only evaluates a single expression. If the expression is true, it executes a single command. If the expression is false, nothing happens. The syntax for this form of if statement is the following.

if (expression) command

The following is an example of a bash or pdksh if statement. This statement checks to see whether there is a .profile file in the current directory:

if [ -f .profile ]

then

 echo "There is a .profile file in the current directory."

else

 echo "Could not find the .profile file."

fi

The same statement written using the tcsh syntax looks like the following:

#

if ( { -f .profile } ) then

 echo "There is a .profile file in the current directory."

else

 echo "Could not find the .profile file."

endif

Notice that in the tcsh example the first line starts with a #. This sign is required in order for tcsh to recognize the file containing the commands as a tcsh script file.

The case Statement

The case statement enables you to compare a pattern with a number of other patterns and execute a block of code if a match is found. The shell case statement is quite a bit more powerful than the case statement in Pascal or the switch statement in C. In the shell case statement, you can compare strings with wildcards in them; you can only compare enumerated types or integer values with the Pascal and C equivalents.

The syntax for the case statement in bash and pdksh is the following:

case string1 in

 str1)

 commands;;

 str2)

 commands;;

 *)

 commands;;

esac

String1 is compared to str1 and str2. If one of these strings matches string1, the commands up until the double semi-colon (;;)are executed. If neither str1 or str2 match string1, the commands that are associated with the asterisk are executed. These commands are the default case condition because the asterisk matches all strings.

The tcsh equivalent of the bash and pdksh case statement is called the switch statement. This statement closely follows the C switch statement syntax. The syntax for the switch statement is the following:

switch (string1)

 case str1:

 statements

 breaksw

 case str2:

 statements

 breaksw

 default:

 statements

 breaksw

endsw

This statement behaves in the same manner as the bash and pdksh case statement. Each string following the case keyword is compared with string1. If any of these strings matches string1, the code following it up until the breaksw keyword is executed. If none of the strings match, the code following the default keyword up until the breaksw keyword is executed.

The following code is an example of a bash or pdksh case statement. This code checks to see whether the first command line option is an -i or an -e. If it is an -i, the program counts the number of lines in the file specified by the second command line option that begins with the letter i. If the first option is an -e, the program counts the number of lines in the file specified by the second command line option that begins with the letter e. If the first command line option is not an -i or an -e, the program prints a brief error message to the screen.

case $1 in

 -i)

 count=`grep ^i $2 | wc -l`

 echo "The number of lines in $2 that start with an i is $count"

 ;;

 -e)

 count=`grep ^e $2 | wc -l`

 echo "The number of lines in $2 that start with an e is $count"

 ;;

 * )

 echo "That option is not recognized"

 ;;

esac

The following is the same example written in tcsh syntax:

# remember that the first line must start with a # when using tcsh

switch ( $1 )

 case -i | i:

 set count = `grep ^i $2 | wc -l`

 echo "The number of lines in $2 that begin with i is $count"

 breaksw

 case -e | e:

 set count = `grep ^e $2 | wc -l`

 echo "The number of lines in $2 that begin with e is $count"

 breaksw

 default:

 echo "That option is not recognized"

 breaksw

endsw

Using Iteration Statements

The shell languages also provide several iteration or looping statements. The most commonly used is the for loop statement. These iterative statements are handy when you need to perform an action repeatedly, such as when you are processing lists of files.

The for Statement

The for statement executes the commands that are contained within it a set number of times. The for statement has two different variations in bash and pdksh. The first form of for statement that bash and pdksh support has the following syntax:

for var1 in list

do

 commands

done

In this form, the for statement executes once for each item that is in the list. This list can be a variable that contains several words separated by spaces, or it can be a list of values that is typed directly into the statement. Each time through the loop, the variable var1 is assigned to the current item in the list until the last one is reached.

The second form of for statement has the following syntax:

for var1

do

 statements

done

In this form, the for statement executes once for each item that is in the variable var1. When you use this syntax of the for statement, the shell program assumes that the var1 variable contains all of the positional parameters that are passed into the shell program on the command line. Typically, this form of for statement is the equivalent of writing the following for statement:

for var1 in "$@"

do

 statements

done

The equivalent of the for statement in tcsh is called the foreach statement. It behaves in the same manner as the bash and pdksh for statements. The syntax of the foreach statement is the following:

foreach name (list)

 commands

end

Again, if this code were the complete program, it should start with a pound sign (and preferably #!/bin/sh to force execution in the Bourne shell). The following is an example of the bash or pdksh style of for statement. This example takes as command line options any number of text files. The program reads in each of these files, converts all of the letters to uppercase, and then stores the results in a file of the same name but with a .caps extension.

for file

do

tr a-z A-Z < $file >$file.caps

done

The following is same example written in tcsh shell language:

#

foreach file ($*)

 tr a-z A-Z < $file >$file.caps

end

The while Statement

Another iteration statement that is offered by the shell programming language is the while statement. This statement causes a block of code to be executed while a provided conditional expression is true. The syntax for the while statement in bash and pdksh is the following:

while expression

do

 statements

done

The syntax for the while statement in tcsh is the following:

while (expression)

 statements

end

The following is an example of the bash or pdksh style of while statement. This program lists the parameters that are passed to the program along with the parameter number.

count=1

while [ -n "$*" ]

do

 echo "This is parameter number $count $1"

 shift

 count=`expr $count + 1`

done

The shift command moves the command line parameters over one to the left (see the following section, "The shift Command," for more information). The following is the same program written in the tcsh language:

#

set count = 1

while ( "$*" != "" )

 echo "This is parameter number $count $1"

 shift

 set count = `expr $count + 1`

end

The until Statement

The until statement is very similar in syntax and function to the while statement. The only real difference between the two is that the until statement executes its code block while its conditional expression is false and the while statement executes its code block while its conditional expression is true. The syntax for the until statement in bash and pdksh is the following:

until expression

do

 commands

done

To make the example that was used for the while statement work with the until statement, all you have to do is negate the condition, as shown in the following code:

count=1

until [ -z "$*" ]

do

 echo "This is parameter number $count $1"

 shift

 count=`expr $count + 1`

done

The only difference between this example and the while statement example is that the -n test command option, which means that the string has non-zero length, was replaced by the -z test option, which means that the sting has a length of zero. In practice, the until statement is not very useful because any until statement that you write can also be written as a while statement. The until command is not supported by tcsh.

The shift Command

The bash, pdksh, and tcsh shells all support a command called shift. The shift command moves the current values stored in the positional parameters one position to the left. For example, if the values of the current positional parameters are

$1 = -r $2 = file1 $3 = file2

and you executed the shift command

shift

the resulting positional parameters would be the following:

$1 = file1 $2 = file2

You also can shift the positional parameters over more than one place by specifying a number with the shift command. The following command shifts the positional parameters two places:

shift 2

This command is very useful when you have a shell program that needs to parse command line options. Options are typically preceded by a hyphen and a letter that indicates what the option is to be used for. Because options are usually processed in a loop of some kind, you will often want to skip to the next positional parameter once you have identified which option should be coming next. For example, the following shell program expects two command line options, one that specifies an input file and one that specifies an output file. The program reads the input file, translates all of the characters in the input file into uppercase, and then stores the results in the specified output file:

while [ "$1" ]

do

 if [ "$1" = "-i" ] then

 infile="$2"

 shift 2

 else if [ "$1" = "-o" ] then

 outfile="$2"

 shift 2

 else

 echo "Program $0 does not recognize option $1"

 fi

done

tr a-z A-Z <$infile >$outfile

The select Statement

The pdksh shell offers one iteration statement that neither bash nor tcsh provides. This statement is the very useful select. statement. It is quite a bit different from the other iteration statements because it does not execute a block of shell code repeatedly while a condition is true or false. What the select statement does is enable you to automatically generate simple text menus. The syntax for the select statement is as follows:

select menuitem [in list_of_items]

do

 commands

done

When you execute a select statement, pdksh creates a numbered menu item for each element that is in the list_of_items. This list_of_items can be a variable that contains more that one item such as choice1 choice2 or it can be a list of choices typed in the command, as in the following example:

select menuitem in choice1 choice2 choice3

If the list_of_items is not provided, the select statement uses the positional parameters just as the for statement does.

When the user of the program that contains a select statement picks one of the menu items by typing in the number associated with it, the select statement stores the value of the selected item in the menuitem variable. The statements in the do block can then perform actions on this menu item.

The following is an example of how you can use the select statement. This example displays three menu items. When the user chooses an item, the program asks whether that item is the intended selection. If the user enters anything other than y or Y, the program redisplays the menu.

select menuitem in pick1 pick2 pick3

do

 echo "Are you sure you want to pick $menuitem"

 read res

 if [ $res = "y" -o $res = "Y" ]

 then

 break

 fi

done

This example introduces a few new commands. The read command is used to get input from the user. It stores anything that the user types into the specified variable. The break command is used to exit a while, select, or for statement.

The repeat Statement

The tcsh shell has an iteration statement that has no equivalent in the pdksh or bash shells. This statement is the repeat statement. The repeat statement executes a single command a specified number of times. The syntax for the repeat statement is the following:

repeat count command

The following example of the repeat statement takes a set of numbers as command line options and prints out that number of periods onto the screen. This program acts as a very primitive graphing program.

#

foreach num ($*)

 repeat $num echo -n "."

 echo ""

end

---

You can rewrite any repeat statement as a while or for statement; the repeat syntax is just more convenient.

---

Using Functions

The shell languages enable you to define your own functions. These functions behave in much the same way as functions that you define in C or other programming languages. The main advantage of using functions as opposed to writing all of your shell code in line is for organization. Code written using functions tends to be much easier to read and maintain and also tends to be smaller because you can group common code into functions instead of putting it everywhere that it is needed.

The syntax for creating a function in bash and pdksh is the following:

fname () {

 shell commands

}

In addition to the preceding syntax, pdksh allows the following syntax:

function fname {

 shell commands

}

Both of these forms behave in the exact same way.

After you have defined your function using one of the preceding forms, you can invoke it by entering the following command:

fname [parm1 parm2 parm3 ...]

Notice that you can pass any number of parameters to your function. When you do pass parameters to a function, it sees those parameters as positional parameters just as a shell program does when you pass it parameters on the command line. For example, the following shell program contains several functions each of which is performing a task that is associated with one of the command line options. This example illustrates many of the concepts covered in this chapter. It reads all the files that are passed in on the command line and, depending on the option that was used, writes the files out in all uppercase letters, writes the files out in all lowercase letters, or prints the files.

upper () {

 shift

 for i

 do

 tr a-z A-Z <$1 >$1.out

 rm $1

 mv $1.out $1

 shift

 done; }

lower () {

 shift

 for i

 do

 tr A-Z a-z <$1 >$1.out

 rm $1

 mv $1.out $1

 shift

 done; }

print () {

 shift

 for i

 do

 lpr $1

 shift

 done; }

usage_error () {

 echo "$1 syntax is $1 <option> <input files>"

 echo ""

 echo "where option is one of the following"

 echo "p -- to print frame files"

 echo "u -- to save as uppercase"

 echo "l -- to save as lowercase"; }

case $1

in

 p | -p) print $@;;

 u | -u) upper $@;;

 l | -l) lower $@;;

 *) usage_error $0;;

esac

The tcsh program does not support functions.

Summary

In this chapter, you have seen many of the features of the bash, pdksh and tcsh programming languages. As you become used to using Linux, you will find that you use shell programming languages more and more often. Even though the shell languages are very powerful and quite easy to learn, you may run into some situations where shell programs are not suited to the problem you are solving. In these cases, you may want to investigate the possibility of using one of the other languages that is available under Linux.